In a software development environment, proper testing is a significant component of the resources required to develop a market acceptable software product. Aggressive development schedules may force trimming of time, human resource, and monetary resources invested in a product. Often, testing phases are accelerated or truncated, rather than sacrificing product functionality already promised to customers, in an effort to maintain cost and ship date milestones. Accordingly, testing may be performed in an ad-hoc, unrepeatable manner. Incomplete testing leads to a product in which many customer execution scenarios have not been exercised, promoting bugs in the released product. Unrepeatable tests require redundant effort to retest upon successive releases or revisions. Such practices tend to allow deployment of a product that is prone to short revision cycles and release of bug fix “patches,” which exacerbate the problem presented by unrepeatable testing.
Scripting methods are known that provide for development of test script files to emulate actual use of a software product. A test script file attempts to codify the inputs and outputs for the operation of a software product, allowing a predetermined input to be associated with an expected output. Aggregation of test script files covering a range of expected operations, along with the expected outputs, allows many test scenarios to be executed by the software product in a relatively short time, mitigating the burdens of manual input and manual determination of a correct result. One such prior art test scripting system is WinRunner™, marketed commercially by MERCURY INTERACTIVE®.
However, a large array of test scripts becomes prone to maintenance issues itself. Revisions to keep test scripts current also require substantial resources. Absence of a unified scripting approach may cause scripts developed by one tester to be unwieldy by another tester, and thus trigger duplication of effort. Incomplete or inaccurate revisions of the test scripts make it unclear whether a failure result is caused by software bugs or merely incorrect test scripts. As a result, software product testing remains an unglamourous, underestimated, and often overlooked, but necessary aspect to software product development.